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EXPLORATION OF AN OCULOMETER -BASED 


MODEL OF PILOT WORKLOAD 


Marjorie J. Krebs, James W. Wingert, and Thomas Cunningham 

Hone3nvell Inc . 

Systems and Research Center 


SECTION I 
SUMMARY 


This Study was performed under contract number NAS 1-13092 sponsored 
by NASA Langley Research Center, Simulation and Human Factors Branch, 
from 29 April 1974 to 31 August 1976. The objective of the study was to 
investigate potential relationships between eye behavior and pilot work- 
load. Ultimately, if stable relationships were found, the goal was to de- 
velop a predictive model usable in avionics display development programs. 
This model would be used for such applications as evaluating the suitabili- 
ty of instrument landing system equipment in providing a Category III 
landing capability for transport aircraft. 

The work was performed by the Honeywell Research Center, Life Sciences 
Group. A Honeywell Mark HA oculometer was used to obtain the eye data 
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in a fixed-base, transport aircraft simulation facility. The data were ana- 
lyzed to determine those parameters of eye behavior which were related to 
changes in level of task difficulty of the simulated manual approach and 
landing on instruments. A number of trends and relationships between eye 
variables and pilot ratings were found. A preliminary equation was written 
based on the results of a stepwise linear regression. High variability in 
time spent on various instruments was related to differences in scanning 
strategy among pilots. A more detailed analysis of individual runs by 
individual pilots was performed to investigate the source of this variability 
more closely. Results indicated a high degree of intra -pilot variability in 
instrument scanning. No consistent workload-related trends were found. 
Pupil diameter which had demonstrated a strong relationship to task diffi- 
culty was extensively re-examined. It was concluded that the generalized 
measure which showed this relationship was most likely not purely pupil 
diameter but a composite index incorporating the influence of other varia- 
bles such as instrument scanning. 

A separate but parallel analysis used maximum likelihood estimation techni- 
ques with added oculometer information to identify dominant system varia- 
bles being used as pilot inputs. The analysis, while limited in scope, was 
considered successful. 

Recommendations for continued analysis of an oculometer -related pilot 
model were provided. 
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SECTION n 


INTRODUCTION 


As the difficulty of the piloting task increases due to adverse weather, air- 
craft stability changes, or a variety of other reasons, the pilot must work 
harder to maintain a constant level of performance. If the resulting work- 
load is too high over too long a period of time, performance decrement will 
very likely result. Furthermore, the higher the workload under routine 
flying conditions, the less reserve the pilot has to draw upon if an emer- 
gency should occur. Clearly, it is desirable to find that workload level 
for the pilot which not only optimizes performance but also provides the 
extra capacity which may be needed only under certain critical conditions . 

In considering alternative methods of measuring workload, it would be of 
value to have a tool with the additional advantage of providing diagnostic 
information as well. That is, it may not be enough to determine that one 
set of flight conditions has a higher workload level than another; the char- 
acteristics of the two conditions which contribute to workload differences 
could be critical. By having this additional diagnostic information, air- 
craft system designers would have an objective basis for eliminating or 
reducing potential problems for the pilot. 

In the present study, the objective was to explore one such workload mea- 
surement technique. Scanning patterns and other eye behavior of pilots 
were recorded over a series of simulated aircraft approach and landing 
maneuvers. Task difficulty was increased by adding turbulence and 
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decreasing aircraft stability. The correspondence between changes in eye 
behavior and changes in task difficulty were then examined as the basis for 
formulating an initial model of visual workload. 

The authors wish to express their appreciation to the following individuals for 
their many contributions during the course of this research program; Mr. 
Doug Engren of Western Airlines, the 737 pilot -consultant, whose exten- 
sive comments and suggestions during the development of the control station 
were invaluable; Mr. James Datta, who modified the simulation software ex- 
tensively for purposes of this study; Mr. Richard Yenni, the Langley Research 
Center test pilot, who provided guidance in the modification of the simulation 
and Cooper-Harper ratings of the flight conditions which helped considerably 
in the analysis of the data; and finally, to Mr. Marvin Waller, Technical 
Monitor at Langley, for his guidance and cooperation during the entire pro- 
gram. 
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SECTION m 


SUMMARY OF RESULTS 


The major results obtained in the study are summarized below. More de- 
tailed information on each of these topics is provided in Section V , 

INSTRUMENT DWELL TIMES 

Very little change was found in average percent time spent on the various 
instruments as a function of task difficulty. Standard deviations reflected 
extremely large variability in the data. This variability was attributed to 
differences in scaiming strategy for individual pilots. Analysis of variance 
performed on percent Attitude Direction Indicator (ADI) dwell time data 
showed no significant effects. 

GENERAL EYE BEHAVIOR 

Four other major variables not specifically related to the particular instru- 
ment set or flight conditions used in the study were investigated. These in- 
cluded: pupil diameter, blink rate, fixation duration, and saccade length. 
While the magnitude of the change in each of these variables as a function of 
flight segment and level of difficulty was small, each one showed a systema- 
tic relationship to the difficulty of the flying task. The change across flight 
segments was larger than that across difficulty levels . 
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WORKLOAD MODEL DEVELOPMENT 


While trends were found in the data, variability was high. Stepwise linear 
regression analysis was explored as a means of expressing workload in eye 
behavior terms. Results indicated a strong relationship between Cooper- 
Harper ratings and general eye behavior. Pupil diameter changes provided 
the highest correlation with pilot ratings and accounted for 89 percent of 
the variance in the response variable. Correlations between pilot ratings 
2 ind time spent on specific instruments were somewhat lower. The pupil 
diameter measure was subsequently analyzed in much greater detail. It 
was concluded that the measure used in the regression analysis was most 
likely not a pure index of pupil diameter but rather a composite measure re- 
flecting both pupil dilation and scanning activity. It was further concluded 
that this in no way diminished its predictive value. 

SYSTEM PERFORMANCE MEASURES 

Three error measures were used to examine total system performance. 
These were RMS errors in glide slope, localizer, and approach speed con- 
trol. When these measures were used to coirpare performance between the 
levels of difficulty, no statistically significant differences were found. 

MAXIMUM LIKELIHOOD ESTIMATION 

The use of maximum likelihood techniques to identify dominant pilot system 
variable inputs was considered successful. Although limited in scope, the 
analysis demonstrated the use of oculometer data as an input to the modeling 
process. 
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SECTION IV 


METHODOLOGY 


OVERVIEW 

Data from a total of 200 runs for terminal approach and landing flight 
segments were collected in a fixed-base, six degree-of-freedom simulator. 
Eight experienced transport pilots from Northwest Airlines were used as 
subjects. Across runs, task difficulty was manipulated by varying aircraft 
stability and adding wind gusts to produce five levels of pilot workload. 

For each run, eye movement data were obtained using a Honeywell Mark IIA 
oculometer. System performance data were computed in the hybrid simulation. 
Eye data, control inputs, and system data were digitized on-line and 
recorded on magnetic tape for later analysis, A variety of plotting techniques 
and statistical analyses were used to evaluate the predictive capabilities 
of these variables in relation to pilot workload. 

SIMULATION 

All data were collected using the XDS Sigma Five Hybrid Computer to 
drive a simulated Boeing 737 aircraft. A general purpose aircraft six 
degree-of-freedom software package (GPASS) was used in the development 
of the simulation for this study. A description of the command laws and 
dynamics of the system are provided in Appendix A. 
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COCKPIT INSTRUMENTATION 


The instrument set used in the study is shown in Figure 1. This set is 
comparable to presently installed instruments in commercial transport 
aircraft. This particular configuration of instruments was chosen to con- 
form as closely as possible to a NASA/Langley Research Center simulation 
which is being used in other eye motion studies. This was the motivation 
for adding the two thin line meters around the ADI. The meter on the right 
side of the ADI displayed raw glide slope deviation. The meter immediate- 
ly below the ADI displayed raw localizer deviation. 

The instrumentation set was familiar to the subject pilots except for two 
features. The ADI had a speed command bug on the left side. This was 
an unfamiliar indicator to the pilots since it was not provided in the Boeing 
727 which they were flying prior to this study. These pilots were accusto- 
med to getting fast -slow information from the airspeed indicator. The ver- 
tical speed indicator consisted of a dial similar to a 737 instrument. This 
turned out to be slightly different from the one used in the 727 and caused a 
few problems in the training sessions for about half of the pilot subjects. 

SIMULATED AIRCRAFT CONTROL STATION 

One of the critical elements of any simulation from the standpoint of its 
impact on pilot performance is the control station. The force -feel system 
was similar in control motions and displacements to the 737, but the dis- 
placements, breakout forces, friction, force gradients, and fluid damping 
were all adjustable over a wide range. Pilots familiar with the 737 aircraft 
worked with the simulator development crew until acceptable parameters of 
the force -feel system were established by trial and error. 
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Figure 1. Instrumentation Set 




EXPERIMENTAL VARIABLES 


The difficulty of the flight control task was manipulated by varying vehicle 
stability and wind conditions. These were varied to obtain a wide range in 
the levels of task difficulty. 

Two sets of vehicle stability models were developed. The first set was the 
baseline 737 vehicle model based on Boeing data. It was then varied as re- 
quired until it was rated as nominally equivalent in handling characteristics 
to a Boeing 737 by two check pilots who were currently flying 737s. 

In the preliminary simulator trials, it was evident that sufficiently difficult 
handling conditions could be generated with wind gust conditions to obtain 
data at the high end of the difficulty scale. Easy handling conditions were 
needed to obtain data at as low a level of difficulty as would be consistent 
with manual flight control conditions. 

The easier handling conditions were generated by changing the stability aug- 
mentation system (SAS) parameters to obtain a more stable vehicle response. 
The autopilot gains changed were those that control the rate feedback com- 
ponents in the stabilization terms of both the lateral -directional and longitu- 
dinal axes of the aircraft. The roll axis SAS gain that is applied to the body 
rate gyro signal used for aileron damping was doubled. To increase this 
component of the pitch damping, the final pitch axis SAS gain that is applied 
to the pitch rate gyro signal was double the nominal value. 

These magnitudes for the final stable vehicle gains were determined by ap- 
plying two criteria: 
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• The pilot workload — as measured by the secondary task workload 
technique — was to be demonstrably lower with the stable vehicle. 

• Pilot ratings of the stable vehicle handling qualities were to be fav- 
orable, with the vehicle as well damped as possible and still res- 
ponsive to pilot inputs. 

The wind model used in this simulation is based on a model recommended 
in Federal Aviation Regulations for the installation approval of automatic 
landing systems in transport category aircraft. This model is documented 
in FAA Advisory Circular 20-57A dated 12 January 1971. 

The wind model used had two modes — steady state and wind gusts. The 
steady state wind mode was entered as a headwind force. Initially the force 
vector was from 330° with respect to nominal heading (0° or north). The 
direction of force changed linearly as a function of altitude down to touch- 
down, At touchdown, direction was 30° with respect to nominal heading. 


The wind gust model was computed as side and upward components. For up 
gusts a force vector component, W , was computed in body axes: 


W 

g 


2V 
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°w 1750 
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X 
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S + 1 


W 
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where 


a = the magnitude of the standard deviation of wind gust magnitude se- 
w 

lected for the trials by the experimenter 
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= aircraft ground velocity 

= a program-generated random number between -1 and +1 
S = the LaPlace operator 


The magnitude of W corresponds to a 20 -knot wind gust (3 a value). 

S 

gusts a force component V was computed in body axes; 
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For side 


where 

= the magnitude of the standard deviation of side wind gust value se- 
lected for the trials by the experimenter 

= aircraft ground velocity 

V = a random number between -1 and +1 
r 

S = the LaPlace operator 

Vgnd = magnitude of ground ride wind 

V, = altitude wind rate (m/sec /m) 
n 

h = altitude (meters) 

The random number generator used was a model which approaches a Gaus- 
sian distribution with mean 0.0 and magnitude + 1.0. 


Gust values of 3 meters per second (l sigma values) were used for both V 
and W . These are higher than expected values for turbulence but were 

O 

established by test pilots in the pre-test phase. These values gave the 
appearance of moderate turbulence to the check pilots. 
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The magnitudes of V and W were chosen experimentally by pretrials using 
pilot subjects who would not participate in the main experiment. The selection 
criteria were; 

1. Realism of the vehicle behavior, and 

2. Range of empirically measured workload 

In order to facilitate interpretation of the final data, it was desirable to main- 
tain the piloting tasks as unchanged as possible throughout the changing levels. 
This eliminated the use of control system augmentation modes which would 
markedly reduce pilot workload but would also materially change the piloting 
taks, as would be the case with a completely automatic mode, for example. 

Once an all-manual control mode decision was reached, the lowest possible 
level of difficulty was defined. It had to be the level obtained when the most 
stable vehicle model was used along with a no-disturbance or zero wind 
flight condition. The most difficult task was determined by making a number 
of simulation runs. This level was defined as the nominal vehicle plus wind 
condition which was marginally acceptable after the pilots were well -prac- 
ticed. The intermediate levels were then determined by trial and error such 
that their workloads, as measured by the secondary workload task, would be 
regularly spaced. 

SELECTION OF FLIGHT CONDITIONS 

Five levels of task difficulty were selected based on the results of a pre- 
liminary test, using two pilots who would not be used in the main experiment. 
These conditions are defined in Table 1. The righthand column of that table 
provides a measure of workload for each condition as defined by the secondary 
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TABLE 1. DEFINITION OF FLIGHT CONDITIONS AND 
ESTIMATED WORKLOADS 


Condition 

Vehicle 

configuration 

Wind 

Workload 
(percent 
time required) 

A 

Stable 

None 

56 

B 

Stable 

Gust 10 

63 

C 

Nominal 

None 

69 

D 

Nominal 

Steady state 

74 

E 

Nominal 

Gust 10 

79 


task. In the pretest, the secondary task consisted of two-digit numbers dis- 
played on a nixie tube located at the extreme left of the instrument panel. 

The pilot was to add the two numbers and report the last digit of their sum. 

It was emphasized that performance on the primary flight task was to be 
maintained and that the secondary task was to be performed on a time -available 
basis only. By counting the average number of reported sums and estimating 
the time required for each one, total time spent on the secondary task could 
be calculated. Total run lenght minus time spent on the secondary task gave 
the time required by the primary flight task. This figure was converted to a p 
percentage for each condition and a workload metric was derived. Thus, the 
higher the number in the "Workload" column of Table 1, the higher the esti- 
mated workload for that condition. The five conditions defined in Table 1 
and used in the main experiment were selected from a larger set of potential 
conditions. The criterion for selection was that the spread in calculated work- 
load be maximized between pairs of conditions and over the entire set of five 
conditions. 
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It should be emphasized that the secondary task was used only during the pre- 
test. In the main experiment the pilot subject's only task was the flight task. 

OCULOMETER 

A Honeywell Mark Ila oculometer was used to record pilot eye data. It per- 
mitted head movement within a cubic foot of space and tracked the eye up to 
+ 40 degrees in the horizontal and +20 to -10 degrees in the vertical dimension. 
It had a measured accuracy across the field of view of approximatly +1 degree 
over the effective range with greatest accuracy near the center. One degree 
is equivalent to about 2.5 cm. on the instrument panel. 

Figure 2 shows the elements of the complete eye -motion recording system. 

It is composed of four major subsystems: 

1. Oculometer - Picks up the image of the pilot's eye and outputs measures 
of eye pointing direction and pupil diameter. 

2. Manual Control Station - Provides a manual control capability so that 
an operator can maintain the pilot's eye within the oculometer field of 
view. 

3. Head Position Monitor - Shows the operator a view of the pilot subject's 
head to assist in reacquiring the pupil image in the event of head 
movement rapid enough to escape from the oculometer monitor. 

4. Video Pointer Assembly - Combines the panel instrument image with 
a superimposed eye marker symbol to indicate the center of gaze of 
the pilot subject. The combination can be viewed during a flight and 
also videotaped for a permanent record. 

A cube approximately 30 cm per side. 
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Figure 2. Functional Block Diagram of Mark Ila Oculometer 













The oculometer was installed in the simulator cockpit with an in-line mount- 
ing in which the optical axis of the oculometer directly viewed the pilot's eye 
through a port in the instrument panel. Either eye of the pilot could be 
tracked. The oculometer was mounted in an adjustable bracket which allowed 
a gross adjustment to be made for eye height of the seated pilot. 

RUN ORDER 

A repeated measures design was used in the main experiment. All subjects 
were given five trials under each of the five conditions. The ordering of 
conditions within a block of five trials was counterbalanced across subjects 
and replications. See Table 2. 

SUBJECTS 

The eight subjects used in the main experiment were all Northwest Airlines 
copilots who were currently flying Boeing 727 commercial transport on a 
full-time basis. 

TRAINING 

The pilot subjects were trained by having them fly the same flight conditions 
that they would fly during the experiment. During the training exercises, the 
system performance variables were measured and recorded. After a block 
of runs was completed, the pilot subjects were given feedback of the perfor- 
mance scores recorded during those runs. Training continued until several 
criteria were met: 
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Table 2. Run Order Assignments 


CONDITIONS 
TASK DIFFI- 
CULTY LEVEL 

SUBJECTS 


1 

2 

3 

4 

5 

6 

7 

8 

A 

52134 

56213 

35124 

25143 

13425 

53142 

34512 

24531 

B 

21453 

14532 

24315 

54132 

32154 

43521 

12354 

42315 

C 

43521 

32145 

12543 

41325 

21453 

25413 

42351 

51234 

D 

35412 

51324 

45321 

13524 

53421 

31425 

13145 

14523 

E 

14235 

24351 

51342 

35241 

43512 

15234 

52413'' 

31425 




• The subjects had settled on a particular set of procedures to use 
for managing flaps, trim, and throttle and always repeated these 
procedures. 

• Performance, defined by errors in glideslope, localizer and airspeed 
as used in the main experiment, had reached a plateau for a 
consecutive set of trial blocks. 

• The pilot was satisfied that he understood the problems and could 
manage them. 

DATA COLLECTED 

The data recorded in the study are listed and described in Table 3. The 
parameters recorded are of a number of types. There are four whose 
purpose is to identify the data: tag, time, range to touchdown, and event 
line. The last needs more detailed description. The mission was broken 
up into segments or "events" which represented different piloting tasks 
and presumably different workloads. The five segments were; 

Event 1 - Straight and Level Flight. Here the pilot task is to main- 
tain a constant altitude while decelerating to final approach 
speed and holding ground track on localizer. 

Event 2 - Glide Slope Acquisition. This is the transition period 
between level flight and constant descent. 

Event 3 - Descent. This is a constant speed, constant flight path 
angle descent. 
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TABLE 3. DATA RECORDED 


Parameter 

Units 

Description 

Tag 


Used to identify the data in each file by 
subject, replication, vehicle stability, 
wind and level 


Time 

Seconds 


Event line 

Arbitrary 

Used to identify the run segments 

Glide slope 

Radians 

Raw data as displayed on the instrument 

Localizer 

Radians 

Raw data as displayed on the instrument 

Velocity 

Knots 

Raw data as displayed on the instrument 

Altitude 

Meters 

Raw data as displayed on the instrument 

Range 

Meters 

Calculated from stop end of runway 

Horizontal pointer 

Centimeters 

Steering information as displayed on ADI 

Vertical pointer 

Centimeters 

Steering information as displayed on ADI 

Speed bug 

Centimeters 

Command information as displayed on 
ADI 

Pitch 

Radians 


Roll 

Radians 


Heading 

Radians 


Throttle 

Kilograms 

Thrust commanded 

Stick F-A 

Radians 

Proportional to yoke fore -aft movement 

Stick R-L 

Radians 

Proportional to yoke right-left movement 

X-eye position 

Centimeters 

Horizontal position of eye relative to 
instrument panel 

Y -eye position 

Centimeters 

Vertical position of the eye 

Pupil diameter 

Millivolts 

Scale factor is 850 mV = 3 mm 

Pen-Up 

Volts 

Level change indicates data loss 
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Event 4 - Preflare, This is the period just prior to flare, commencing 
at an altitude of 45. 12 meters and ending when the flare com- 


mand starts. 

Event 5 - Touchdown. This segment includes flare and ends at touch- 
down. 

Three major indicators of total system performance were recorded. These 
are the errors from nominal in glide slope, localizer, and velocity and 
were used as the criteria variables to characterize performance of the 
pilots on the five levels of difficulty. 
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SECTION V 
RESULTS 


OVERVIEW 

Both system and eye analog data were sampled at a rate of 30 times per 
second, digitized, and placed on magnetic tape for later analysis . System 
data were analyzed to relate changes in performance to changes in task 
difficulty and to evaluate the difficulty index established in preliminary tests 
prior to the main experiment. 

The eye data were analyzed to determine what, if any, systematic changes 
in eye behavior occurred as a function of task difficulty. The ultimate goal 
of this analysis was to define a set of eye behavior measures correlated 
with changes in pilot workload. These measures were then evaluated as to 
their applicability to a workload related model. Linear regression tech- 
niques were used to explore general trends in the data. Short-time history 
plots were used to examine individual run data in more detail. Finally, 
samples of data taken during prolonged dwells on the ADI were examined 
using maximum likelihood estimation techniques. The goal was to examine - 
the value of this technique in determining what information the pilot was 
using when he looked at the composite flight director. 

DATA REDUCTION 


The series of x and y coordinates representing eye position were analyzed 
over time to determine when the subject dwelled (fixated) on some position 
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in the instrument array. A consecutive series of four x-y values which 
varied less than one degree from each other was defined as a minimum 
fixation time. Additional samples were added to a fixation until changes in 
the x-y values exceeded the one-degree limit. The number of samples con- 
tained in one of these fixations defined its duration. The average value of 
all the samples in the set determined its location. When a pair of x-y 
values exceeded the one -degree circle, it was counted as the beginning of 
an eye movement (saccade). A saccade continued until a new fixation was 
determined. The distance between two fixations defined the saccade length. 
This auialysis, performed on the computer, was done for all samples of eye 
data across all 200 trials. 

The fixation locations were then related to the specific flight instruments by 
defining the outer boimdaries of each instniment and then counting any fixa- 
tion within those boundaries as a dwell on that instrument. The number of 
samples contained in one fixation determined the fixation duration or dwell 
time. The sum of these over a specified interval was the total dwell time. 

In addition to instrument dwell time, fixation duration, saccade length, mea- 
sures of pupil diameter, and blink rate were also obtained. For each run, 
a summary matrix of inter- and intra -instrument fixations was generated. 
This "fixation transition matrix" summarized the percentage of fixation 
shifts which resulted in either a change from one specified instrument to 
another or changes in fixation within the same instrument. 

SYSTEM PERFORMANCE DATA ANALYSIS 

The statistically analyzed system performance data parameters were the 
aircraft deviations from commanded profile: 
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glide slope errors (meters) 
localizer errors (meters) 
velocity errors (meters per second) 

Each of these parameters was analyzed to determine if there were differences 
in performance on the individual conditions. This is important since per- 
formance can be traded off against workload. That is, if a condition is 
more difficult to control, the pilot can accept greater error and still have 
about the same workload as in an easier condition where error is lower. 

System performance was compared between levels to determine if there 
were different accuracies obtained for the differing task difficulties. When 
the subjects are pooled, only small differences in mean error result. Error 
data for Event 3, the descent segment, are presented in Table 4. 


Table 4. Means of Error Data for Event 3, the 
Descent Segment (Standard Deviations 
in Parentheses) 



CONDITION 


A 

B 

C 

D 

E 

• Glide Slope Error 
(meters) 

23.85 

(13.05) 

22.23 

(10.10) 

23.58 

(12.57) 

16.78 

(10.23) 

25.47 

(12.17) 

Localizer Error 
(meters) 

4.03 

(1.04) 

3,81 

(1.13) 

5.40 

(1.22) 

5.40 

(1.65) 

4.85 

(1.13) 

Speed Error 
(meters per sec) 

2.90 

(1.71) 

2.84 

(1,86) 

2.96 

(1.77) 

2.87 

(1.60) 

2.53 

(1.80) 
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Individual comparisons of these error data by subject and level of difficulty 
were made using the Tukey Test. This method was selected from among 
several which have been developed to minimize errors when a large number 
of repeated comparisons are made. It has been established that wrong de- 
cisions owing to type 1 error are probable when testing repeatedly at a 
fixed a level. 


A number of statistical researchers have developed methods for constructing 
simultaneous confidence intervals which avoid the pitfall of permitting the 
type 1 error to become excessively large. These methods basically depend 
on the use of a statistic calculated from the range of the means of the com- 
parisons to be made, rather than using all the treatment means (as in the 
F tests). This kind of technique is more appropriate to the present data 
because the data distributions tend to pile up at certain values rather than 
distribute across the range. The statistic calculated is; 


q = 


highest lowest 
V Mean Square Error /n 


( 1 ) 


The classic method (Newman -Keuls) prescribes a critical value for the dif- 
ference between two means that depends on the number of ordinal steps be- 
tween the two means being compared. The Neuman-Keuls procedure was 
modified by Tukey to use a single critical value for all comparisons, rather 
than adjusting the critical value depending on the number of steps between 
the treatment means being compared. This approach is more conservative 
in keeping the type 1 error small because it uses the critical value for the 
maximum number of steps for all comparisons. This critical value is, 
therefore, the largest criterion number to satisfy. This assures that the 
procedure has the property that all tests performed on differences between 
pairs of means have a level of significance which is at most equal to a . 
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The results of the Tukev tests for individual comoarisons were not satisti- 
cally significant at the a » 0.05 level. There are not, therefore, consistent 
degradations of performance in the more difficult flight conditions. This 
indicates that pilot workload must be increasing in the harder flight condi- 
tions to compensate for the increasing difficulty of the control task. 

Analysis of Vehicle -Related Data 


As indicated above, pilot performance, in terms of absolute error of devia- 
tion scores, did not change significantly across levels of difficulty. While 
overall error was not statistically different across levels, the amount of 
change reflected in the instruments and pilot control maneuvers required to 
maintain performance errors at a fairly constant level varied widely. In 
Table 5, the computed rate of change of seven state variables and three 
control input variables are presented as a function of task difficulty. The 
first two columns, HSP and VSP, represent rate of change of the horizon- 
tal and vertical command bars of the ADI. Theunits are centimeters /second. 
Airspeed (SPD) variability is in meters /second. Pitch, roll and heading 
are in radians /second and altitude (ALT) is in meters /second. The three 
control movements are throttle (kilograms /second), stick movement fore 
and aft (radians /second) and stick movement right and left (radians /second). 

Table 5 shows that the change rates are not ordered according to the preas- 
signed ordering of levels of task difficulty. In fact, if the data are reordered 
as follows; A, C, D, B, E from "easy” to "difficult", almost all of the state 
and control variable values in the table are also ordered from low to high. 
Figures 3, 4 and 5 show six of the variables from Table 5. 
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Table 5. RMS Rate of Change 
Seven State Variable 


CONDITION 

HSP 

cm/ 

sec 

VSP 

cm/ 

sec 

SPD 

meters/ 

sec 

A 

0.77 

0.36 

0.20 

B 

1.35 

0.54 

0.28 

C 

1.00 

0.42 

0.20 

D 

0.88 

0.46 

0.23 


PITCH 

rad/ 

sec 

.00 

.00 

.00 

.00 


E 


1.57 


0.64 


0.31 


.01 


IS a Function of Task Difficulty for 
! and Three Control Inputs 


ROLL 

rad/ 

sec 

HEAD 

rad/ 

sec 

ALT 

meters/ 

sec 

THROT. 

kg/hr/ 

sec 

.01 

.01 

0.17 

15.62 

.01 

.02 

0.23 

22.71 

.01 

.01 

0.21 

15.98 

.01 

.01 

0.06 

17.10 

.02 

.02 

0.24 

25.37 






















RATE OF CHANGE (CENTIMETERS/SEC) RATE OF CHANGE (CENTIMETERS/SEC) 




Figure 3, Rate of Change of the Horizontal (Top) and 

Vertical (Bottom) Steering Pointers of the ADI 
as a Function of Reordered Task Difficulties 
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Figure 5. Rate of Change in Throttle (Top) and Stick Activity 

(Bottom) as a Function of Reordered Task Difficulties 
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A test pilot from Langley Research Center provided Cooper-Harper ratings 
(Appendix B) for each of the five conditions. Table 6 shows that these ratings 
are perfectly correlated with reordered difficulty levels shown in Table 5, The 
intervals between the Cooper-Harper ratings provided by the NASA test pilot 
were also highly predictive, particularly the relative similarity between condi- 
tions A, C, and D. Figure 5 clearly shows this for pilot control movements. 

Table 6. Comparison of Cooper-Harper Ratings of NASA 
Pilot and Reordered Task Difficulties 


Task Destination 

Cooper-Harper Rating 

Difficulty Level 

A 

3.0 

1.0 

B 

5.5 

4.0 

C 

3.5 

2.0 

D 

4.0 

3.0 

E 

7.0 

5.0 


Task Difficulty and Pilot Workload Redefined 


The above findings for the system data indicate that the preliminary ordering 
of levels of task difficulty using the secondary task method was not predictive 
of the ordering obtained in the main experiment. Since all of the system vari- 
ables and the pilot control inputs are ordered in a way perfectly correlated 
with the Cooper-Harper ratings, this latter measure was chosen as the index 
of pilot workload. The experimental data discussed below are interpreted 
using the reordered task difficulty levels as indicated in Table 6. 

Analysis of Oculometer Data 


The oculometer data were examined in two different ways. First, general 
trends were investigated using linear regression techniques. For this purpose, 
data were averaged over each run for each of the variables of interest. Follow- 
ing this general analysis, individual runs by individual pilots were examined in 
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much greater detail. The primary oculometer variables investigated were: 

1) Average dwell time (fixation duration) 

2) Total dwell time on each instrument (per run) 

3) Saccade length 

4) Blink rate 

5) Pupil diameter 

LINEAR REGRESSION ANALYSES 

In an attempt to build a preliminary model of visual workload, all of the eye 
variables listed above were analyzed using stepwise linear regression tech- 
niques. In the regression analysis the correlation between changes in 
task difficulty and the set of performance measures was computed. If the 
correlations between task difficulty and some subset of performance 
measures were strong enough, the output of the analysis would be an 
expression of the relationship between the two. Ideally, the relationship 
is descriptive because it accounts for a large percent of the total variability 
obtained in the experiment and thus be highly descriptive. 

The results of this analysis are shown in Table 7, a correlation matrix 
showing the relationship between eye variables and pilot rating, and in 
Table 8, a summary of the stepwise linear regression analysis. Variables 
4 through 15 in the correlation matrix represent percent total dwell time on 
the specific instrument. The correlation matrix indicates that the oculo- 
meter variable with the highest correlation with the predictor variable was 
pupil diameter (0.94), followed by saccade length (0.88) and percent total 
ADI dwell time (0. 79). The remaining variables had moderate to low cor- 
relations with pilot rating. 
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Table 7 


Correlation Matrix for All Eye Variables Entered into the Re- 
gression Analysis. The Bottom Row of This Table Indicates 
the Correlation of Each of these Variables with the Predictor 
Variable; Pilot Rating. 


VARIABLES 

CORRELATION 

1. 

Pupil Diameter 

— 

1.000 

.436 

. 921 

.110 

.570 

. 294 

.602 

.469 

.448 

. 364 

.458 

.499 

.183 

. 320 

. 858 

. 944 

2. 

Fixation Duration 

.436 



. 034 

. 214 

. 105 

.354 

. 178 

. 165 

. 123 

.175 

. 205 

. 050 

, 097 

- 400 

. 440 

3. 

Saccade Length 

.921 

.445 

1. 000 

. 199 

. 650 

. 324 

. 507 

.479 

. 468 

. 443 

. 544 

. 573 

. 179 

. 343 

. 768 

. 880 

4. 

Clock 

.110 

. 034 



.222 

.014 

.080 

.024 

.029 

.150 

. 058 

. 056 

. 000 

. 041 

. 025 

.116 

5. 

Airspeed 

. 570 

. 214 

.650 

.222 

1.000 

. 169 

.427 

.292 

.230 

. 174 

.255 

. 341 

.060 

. 138 

. 354 

. 541 

6 

Bank 

.294 

. 105 

.314 

. 014 

. 169 



.241 

.031 

.040 

. 131 

. 102 

.263 

. 174 

.264 

. 258 

7. 

Speed Bug 

. 602 

.354 


.089 

.427 

. 067 

1.000 

. 192 

.217 

. 123 

. 097 

. 250 

.008 

.040 

.243 

. 624 

a. 

Glide 3 lope 

.469 

.178 • 

.479 

.024 

.292 

.241 

.192 

1.000 

. 116 

. 117 

.386 

.278 

. 120 

.080 

.410 

.413 

9. 

Localizer 


. 165 

.468 



.031 

.217 



.441 

i— 


.013 

. 203 

.213 

. 469 

10. 

HSI 

.364 


. 441 

. 150 

. 174 

.040 

. 123 

. 117 

.441 

1.000 

. 163 

. 344 

.018 

. 169 

. 163 

.382 

11. 

Altimeter 

.458 





.131 

.097 



n 


. 344 

. 133 

.219 

. 439 

.412 

12. 

VSI 

.499 

. 205 

. 573 

. 056 

.341 

. 102 

. 250 

.278 

.346 

1 

. 344 

.344 

1. 000 

.093 

. 179 

.336 

. 464 

13. 

Flaps 

. 183 



.000 

.060 

.263 


. 120 


1— 

. 233 

. 093 

1.000 

. 115 

. 186 

. 165 

14. 

Fuel 

. 320 

.097 

. 343 

. 041 

. 138 

. 174 

. 040 

.080 

.203 

. 169 

. 219 

.179 

.115 

1. 000 

.294 

. 262 

15. 

ADI 

. 858 

, 400 

, 768 

.025 

.354 

.264 

.243 

.410 


. 163 

.439 

. 336 

. 186 

. 294 

1.000 

. 793 

16. 

Pilot Rating 

1 

. 944 

. 440 

. 880 

.116 

. 541 

.258 

.624 

.413 

.469 

. 382 

.412 

. 464 

. 165 

.262 

. 793 

1.000 


CO 

CO 
















































































































































































































Table 8 shows that pupil diameter alone accounts for 89 percent of the vari- 
ance in pilot rating as indicated by the R value. The remaining variables 
when combined with pupil diameter contribute little to the predictive power, , 
Saccade length drops out of the list of variables because of its high correla- 
tion (0.92) with pupil diameter. 

This relationship can be expressed in the following equation: 

Workload = 2.34 x pupil diameter (2) 

where workload is the value of the Cooper-Harper pilot rating for that con- 
dition and pupil diameter is in millimeters. 

While pupil diameter was initially considered to have some potential in its 
relationship to pilot workload, the fact that none of the other variables en- 
tered into the equation was surprising. Therefore, each of the oculometer- 
based measures was examined in more detail. 

INSTRUMENT DWELL TIMES 

The percent of the total dwell time spent on each of the ten major instru- 
ments was computed by subject for each level of task difficulty. These data 
were then pooled across subjects to obtain an overall average percent total 
dwell time per instrument for each of the five difficulty levels. The results 
are presented in Table 9 together with the associated standard deviations. 
The average values in Table 9 show trends for the ADI and VSI but the re- 
maining instruments show no consistent increase or decrease. The most 
striking feature in this table is the magnitude of the standard deviations. In 
many cases, they are several times larger than their corresponding means. 
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Table 8. Summary Table of Stepwise Linear Regression Analysis Show- 
ing the Sequence in Which Variables Were Entered, R^ (the 
Percent of Variance Accounted for), and Increase in (the 
Increase in Predictability Achieved with the Addition of a 
Given Variable). 


Step 

Variable 

Multiple 

Increase 

Number 

Entered 

Removed 

R 

R 

In R"^ 

1 

Pupil Diameter 


. 9436 

. 8903 

. 8903 

2 

Speed Bug 

- 

. 9461 

. 8952 

. 0043 

3 

Localizer 

- 

. 9479 

. 8985 

.0033 

4 

ADI 

- 

. 9500 

. 9024 

. 0040 

5 

HSI 

- 

. 9526 

. 9075 

. 0050 

6 

Airspeed 


. 9536 

. 9094 

.0019 

7 

Bank 


. 9540 

. 9102 

. 0008 

8 

Altimeter 

- 

. 9544 

. 9109 

. 0007 

9 

Flaps 

- 

. 9547 

. 9115 

. 0006 

10 

Fuel 

- 

. 9549 

. 9118 

. 0003 

11 

Clock 

- 

. 9551 

. 9121 

. 0003 

12 

VSI 

- 

. 9551 

.9123 

.0002 

13 

Glideslope 

- 

. 9551 

. 9123 

.0000 

14 

Fixation Duration 

- 

.9551 

.9123 

.0000 





TASK 

DIFFI- 

CULTY 

B 

ASP 

GSL 

LOC 

Level 1 

.05 

(.20) 

6.24 

(6.92) 

1.05 

(2.57) 

7.45 

(13.15) 

Level 2 

.09 

(.38) 

7.82 

(13.09) 

.88 

(1.27) 

6.49 

(11.73) 

Level 3 

.32 

(2.57) 

5.45 

(7.04) 

.79 

(1.61) 

8.58 

(15.05) 

Level 4 

.12 

(.65) 

5.39 

(6.47) 

1.06 

(1.80) 

7.42 

(13.01) 

Level 5 

.22 

(1.59) 

5.70 

(7.31) 

.67 

(1.34) 

9.37 

(.18.06) 


INSTRUMENT 


HSI 

ALT 

VSI 

FLAPS 

FUEL 

ADI 

3.50 

2.05 

4.30 

.04 

.36 

74.95 

(6.94) 

(4.07) 

(8.14) 

(.21) 

(1.03) 

(58.44) 

2.81 

2.46 

3.39 

.07 

.30 

75.69 

(4.24) 

(5.40) 

(4.12) 1 

! 

(.51) 

(.89) 

(61.54) 

5.33 

1.95 

3.09 

.08 

.49 

73.93 

(13.06) 

(3.62) 

(3.18) 

(.45) 

(1.54) 

(63.22) 

3.48 

1.91 

3.06 

.12 

.29 

77.14 

(7.68) 

(2.90) 

(6.31) 

(.41) 

(.85) 

(64.89) 

5.80 


2.36 

1 

.13 

.15 

74.10 

(14.61) 


(3.17) 

(.95) 

(.43). 

.(.68.54) 





























































Prior to performing an analysis of variance on the dwell time data, each of 
the columns in Table 9 was tested for homogeneity of variance using Cochran's 
test (Reference 5, Myers, 1966). This statistic uses the ratio of the sums 
of the variances for a given set of data to the maximum variance from that 
set. The obtained vsdue is then compared to a critical value. If the critical 
value is exceeded, the variances are not homogeneous and a critical assump- 
tion of the F-test is violated. The results of that analysis indicated that of 
the data for the ten instruments, only the ADI data demonstrated homogeneity 
of variance. An analysis of variance was computed for these data and a 
summary of that analysis is provided in Table 10. Neither the two main ef- 
fects (level and replication) nor their interaction was significant. 

One possible reason for the lack of significant tr 2 nds and also the high vari- 
ability discussed above is that individual pilots had different scanning stra- 
tegies. It was also possible that these individual strategies might reflect 
some consistent effects produced by increasing task difficulty. In order to 

investigate this possibility, each of the 200 runs was examined in detail, 

2 

looking for a relationship between instrument scanning and workload. 

Several methods were used to examine the individual run data. First, averaged 
data for each run by each subject was examined. Second, short time history 
plots were prepared for each of the 200 runs. These plots, generated on the 
computer, were examined to search for clues concerning what changes, if 
any, occurred in pilot scanning as task difficulty increased. An example of 
one series of plots is provided in Figures 6a, 6b, 6c, and 6d. The size of 


The subjects were very similar in background. There was no available 
evidence suggesting that one or more could be classified as either substan- 
tially more or less experienced than the others. 
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Table 10. Summary Table, Analysis of Variance=i= for 
the Percent Total Time Spent on the ADI as 
a Function of Task Difficulty 


Source 

SS 

' df 

MS 

F 

Total 

85379.00 

199 



Level (L) 

242.95 

4 

60.74 

0.47 — 

Replication (R) 

1675.11 

4 

418.78 

1.19 - 

RxL 

589.48 

16 

36.84 

0.54 - 

Subjects (S) 

61642.14 

7 

8806.02 


SxL 

3655.68 

28 

130.56 


SxR 

9874.76 

29 

352.67 

-- Non 

SxRxL (Residual) 

7698.88 

112 

68.74 

Significant 


^The model used for this analysis was the repeated measures, randomized 
blocks design as discussed in Myers (Reference 5, 1966, pg. 156). In the 
non-additive model assumed here, the appropriate F -ratios were con- 
structed as follows: 




MS_ , IMS- _ , 
RxL SxRxL 
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Figure 6b, Instrument Fixations, Subject 1, Difficulty Level 3, Replication 1 



each plot is greatly reduced from the original. The ten instruments are 
represented on the ordinate and dwells on these instruments are plotted as 
a function of time into run; thus both frequency and duration information is 
available. It would be prohibitive in terms of space to attempt to reproduce 
all of these plots and would add little information to this report; however, it 
can be stated that differences were difficult to quantify. What was seen was 
a difference in the overall pattern of the line structure in the plot. In addi- 
tion to differences among pilots, there were also differences observed 
within a pilot over several replications of the same difficulty level (see 
Figure 6a, 6b, 6c, and 6d). That is, a consistent pattern representative of 
one pilot or one group of pilots was not found. 

The variability seen in the plots was explored further by looking at examples 
of inter- and intra-subject data. In Figure 7, the average number of dwells 
on the ADI is plotted as a function of task difficulty for six subjects. Both 
the group mean and individual data are shown. While the group mean tends 
to rise slightly as task difficulty increases, individual data show different 
trends. Data for Subjects 2 and 6 show a decrease in number of dwells on 
the ADI; for Subject 4, an increase; and for Subject 1, no consistent trend. 
Thus the group trend is not generally representative of the individual means. 
Similar results were obtained for other members of the instrument set. 

As a final attempt to discover a consistent relationship between instrument 
scanning and task difficulty, a series of stepwise linear regressions analyses 
were run on subsets of the data. Each run was divided into eight to ten 30- 
second segments (depending on run length). Averages were obtained for each 
segment for each instrument for the following variables; 
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NUMBER OF FIXATIONS PER RUN 


SUBJECT 3 



1 2 3 4 5 

LEVEL OF DIFFICULTY 


SUBJECT 4 

GROUP 
SUBJECT 2 


SUBJECT 1 

■ GROUP MEAN 
----SUBJECT MEAN 


SUBJECT 6 


Figure 7. Number of Fixations per Run on the ADI 
Individual and Group Means 



1) Number of looks 


2) Average dwell time 

3) Total dwell time 

Regression analyses were, then run for all subjects, for individual subjects, 

and for all subjects pooled over selected time segments (for example, the 

last 90 seconds before touchdown). In all cases, the response variable was 

the Cooper-Harper rating. In all, about 30 such analyses were performed. 

2 

None, however, yielded total R values above 0.35 and most were considera- 
bly below that value, even when all variables were forced into the equation. 

At least as far as the present data set is concerned, it must be concluded 
that no relationship has been found between instrument scanning activities 
and pilot workload. 

Average values on a number of other eye-related variables were obtained 
for each subject on each run. It was later decided that the averages over 
an entire four or five minute trial were not discriminating across some of 
the variables; therefore, each run was further divided into five segments 
as follows; 

Event 1 - Straight and level flight 
Event 2 - Glideslope acquisition 
Event 3 - Descent 
Event 4 - Preflare 
Event 5 - Touchdown 

Of these five Events, only 1, 3, and 5 proved to be useful, since glideslope 
acquisition and preflare lasted only a few seconds --an interval too short 
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) 

to meaningfiilly express data in percentages. Therefore, naost of the data 
in the following section are presented as averages over the entire run, or 
averages for Events 1, 3, and 5. Data from Events 2 and 4, representing 
less than two percent of the total, have not been used, 

SACCADE LENGTH 


Under higher workload levels, it was hypothesized that a subject might tend 
to "lock on" to an individual instrument or he might develop an erratic scan 
pattern. Either of these, if they were to occur, would be reflected in the 
average saccade length or interfixation distance. 

An erratic scan pattern would likely lead to longer average saccade lengths. 
On the other hand, if the pilot were to focus heavily on one instrument or 
several central instruments, it would lead to much shorter average distances. 
The average saccade lengths for individual subjects by event and level are 
presented in Table 11 and Figure 8. There is a slight but consistent de- 
crease in length of eye movement across levels and a slightly larger de- 
crease across events. In fact, comparisons across events within subjects 
showed that 86 percent of the saccade lengths decreased from Event 1 to 
Event 3 and Event 3 to Event 5 . 

SACCADE LENGTH VERSUS INSTRUMENT SCANNING 

It was obvious from studying the instrument scan data averaged across sub- 
jects that variability was much too high to make generalizations. Therefore, 
a more detailed analysis of one individual subject was performed to help 
clarify some of the potential relationships. In particular, the relationship 
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Task Difficulty 


Figure 8. Average Change in Saccade Length as a Function 
of Changes in Task Difficulty 
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Level 1 


Level 2 




Event 


— 

Event 


Subject 

1 

3 

5 

1 

3 

5 

1 

— 

5.77 

3.40 

— 

6.20 

3.56 

2 

4.22 

3.15 

2.92 

4.67 

2.87 

2.62 

3 

4.93 

2.74 

1.88 

4.09 

2.31 

1.24 

4 

3.96 

3.86 ■ 

2.74 

4.22 

3.56 

2.82 

5 

3.66. 

3.45 

2.01 

3.78 

3.99 

2.18 

6 

6.60 

4.72 

4.27 

6.43 

4.45 

3.66 

7 

5.21 

4.72 

4.45 

4.93 

5.08 

3.48 

8 

4.50 

3.86 

3.96 

4.32 

4.11 

4.95 

Average 

4.72 

4.04 

3.20 

4.62 

4.06 

3.07 



Level 3 



Level 4 



Level 5 



Event 



Event 



Event 


1 

3 

5 

1 

3 

5 

1 

3 

5 

— 

6.12 

4.09 

— 

6.17 

4.04 

— 

5.97 

3.61 

4.32 

3.33 

2.51 

4.52 

2.92 

3.96 

3.45 

2.36 

3.38 

3.56 

2.31 

1.47 

4.52 

2.11 

1.42 

3.53 

2.03 

1.63 

3.96 

3.18 

3.25 

3.63 

3.20 

3.43 

3.45 

2.87 

2.74 

4.04 

3.30 

2.08 

3.25 

3.56 

1.93 

3.76 

3.73 

2.18 

5.49 

4.47 

3.68 

6.63 

4.72 

4.29 

5.51 

4.47 

3.76 

5.69 

4.75 

4.27 

5.66 

4.50 

3.02 

5.36 

4.37 

3.68 

4.85 

3.89 

3.61 

4.47 

3.78 

5.03 

3.53 

3.61 

5.08 

4.55 

3.91 

3.12 

4.67 

3.86 

3.07 

4.09 

3.68 

3.25 

















between saccade length and certain measures related to the ADI was investi- 
gated. Data for Subject 3 averaged only over replications of each condition 
is presented in Table 12. This particular subject was chosen because he 
demonstrated large decreases in saccade length across events and levels as 
compared to the other subjects. 

In the first row of Table 12, decreases from Event 1 to Event 5 averaged as 
much as three centimeters. The question was: is this decrease due to 
more time spent on one instrument (the ADI) or to more transitions within 
the ADI, or to less scanning of peripheral instruments? 

Note in Table 12 that ADI dwell time shows only a slight increase, the per- 
centage of fixations a slightly greater increase, but the percentage of fixa- 
tion transitions within the ADI (as opposed to cross checking other instru- 
ments) shows a marked increase across events. In addition, the percen- 
tage of within -ADI transitions for Event data increases across levels more 
than do the other measures. 

The last row in Table 12 reflects the percentage of fixation transitions be- 
tween the ADI and the instrument indicated. In each case, the instrument 
noted is the one most often fixated immediately before or after the ADI, 
typically the airpseed indicator. Adding the numbers in the last two rows 
together within a column yields totals very close to 100 percent in all 
cases for Event 5 data across the different difficulty levels. The issue of 
whether this is "appropriate" behavior in terms of relative amounts of in- 
formation available in the various instruments or whether it demonstrates 
some evidence of the "locking on" phenomenon remains a question. 
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Table 12. Relationship between Saccade Length and Instrument Scan- 
ning Behavior for Selected Instruments, Data for Subject 3 


Fixations 
on ADI 


Transi tions 
Within ADI 


Transitions 

Between 


1 

Level 1 

Event 

3 

5 

1 

Level 2 

Event 

3 

5 

1 

Level 3 

Event 

3 

5 

1 

Level 4 

Event 

3 

5 

4.93 

2.74 

1.88 

4.09 

2.31 

1.24 

3.56 

2.31 

1.47 

4.52 

2.11 

1.42 

81.2 

90.8 

96.8 

86.4 

94.1 

98.9 

86.5 

94.6 

98.4 

85.0 

95.5 

97.3 

73.5 

83.5 

90.9 

79.1 

87.7 

96.4 

80.2 

88.4 

39.3 

79.8 

90.9 

93.9 

58.6 

73.9 

83.9 

66.5 

80.0 

93.7 

67.8 

80.9 

95.1 

65.3 

84.9 

91.9 

9.2 

4.1 

7.0 

6.1 

9.6 

3.6 

11.4 

6.5 

2.8 

8.2 

7.3 

3.6 

VSI 

ASP 

ASP 

VSI 

ASP 

ASP 

VSI 

ASP 

ASP 

VSI 

ASP 

ASP 


Event 

1 3 5 


3.53 2.03 1.63 







Because of the consistency found in Table 12 for within ADI transitions for 
Subject 3, similar data for all eight subjects were examined. The percentage 
of changes in fixation which began and ended on the ADI were included. These 
data were then averaged by level of difficulty and by flight segment (Event). 
Similar data were obtained for all the remaining instruments. Together, 
these numbers represent the sum of the diagonal of the fixation transition 
matrix. Both ADI transitions and all other withinrinstruments transitions 
are plotted in Figure 9 by level of difficulty and by event. As can be seen 
in Figure 9a, the percentage for both data sets rises slightly and fairly con- 
sistently as the level of task difficulty increases. From Level 1 to Level 5, 
the range is about five percent. In Figure 9b, a larger increase (about ten 
percent) is obtained across flight segments. While the changes are not as 
large as those reported in Table 12 for a single subject, they are reasonably 
consistent. Thus, the data suggest that the probability that the subject will 
remain on a given instrument increases as task difficulty increases. 

FIXATION DURATION 

If either the erratic scan pattern or locking -on behavior discussed above 
were to occur at higher workload levels, it would also likely result in no- 
ticeable changes in average fixation duration. Average fixation durations 
pooled across subjects as a function of task difficulty are shown in Figure 
10 and by level and event for individual subjects in Table 13. Across levels, 
fixation duration shows a rather unsystematic trend. Within levels, the 
consistent trend is toward longer fixations as the pilot shifts from Straight 
and Level Flight (Event 1) to Flare and Touchdown (Event 5). This increase 
occurs in 75 percent of all comparisons; however, the differences are small, 
averaging one -tenth of a second or less. The increase may be due to the 
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Figure 9a, b. Percentage of Transitions Within a Given Instrument (Sum 
of the Diagonal of the Transition Matrix) by Event and by 
Level of Difficulty 
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.37 


.35 


.33 


.31 



Task Difficulty 


Figure 10. Average Change in Fixation Duration 
as a Function of Task Difficulty 
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Table 13. Average Fixation Duration by Subject for Each Event 
and Difficulty Level. 



Level 1 . 

Level 2 

Level 3 

Level 4 

Level 5 



Event 



Event 



Event 



Event 



Event 


Subject 

1 

3 

5 

1 

3 

5 

1 

3 

5 

1 

3 

5 

1 

3 

5 

1 

-- 

.28 

.23 

— 

.29 

.27 

— 

.27 

.30 

— 

.25 

.30 

— 

.29 

.30 

2 

.24 ■ 

.27 

.26 

.24 

.28 

.27 

.26 

.29 

.33 

.24 

.28 ' 

.24 

.28 

.29 

.30 

3 

.29 

.41 

.56 

.34 

.44 

.65 

.32 

.47 

.57 

.30 

.41 

.47 

.30 

.47 

.61 

4 

.28 

.33 

.42 

.28 

.35 

.46 

.29 

.37 

.39 

.29 

.37 

.35 

.30 

.39 

.44 

5 

.44 

.33 

.41 

.38 

.36 

.58 

.30 

.30 

.42 

.30 

.34 

.45 

.47 

.32 

.47 

6 

.26 

.24 

.32 

1 

.26 

.25 

.38 

.26 

.24 

.33 

.23 

.24 

.33 

.25 

.28 

.37 

7 

.28 

.27 

.29 

.29 

.24 

.28 

.26 

.25 

.30 

.26 

.28 

.40 

.23 

.29 

.36 

8 

.22 

.22 

.22 

.23 

.23 

.22 

.22 

.24 

.25 

.25 

.27 

.23 

.20 

.21 

.24 

Average 

.29" 

.29 . 

.34 

.29 

.31 

.39 

.27 

.30 

.36 

.27 

.31 

.35 

.29 

.32 

.39 

























fact that the pilot is seeking more rate information as he approaches touch- 
down. It is interesting to note, however, that both saccade length and fix- 
ation duration data exhibit trends which are suggestive of the locking -on hy- 
pothesis. The fact that the changes are more marked across events than 
across levels implies that either the change in workload was greater within 
a run as touchdown was approached, or, as was suggested above, it merely 
reflects a change in the type of information the pilot is using as he approaches 
touchdown, that is, more rate information. 

BLINK RATE 

Blink rate data, unlike the other measures, were not obtained from the digi- 
tized flight data. Video tapes of the pilot's eye were obtained for each of 
the 200 runs. When the pilot blinked, it was clearly indicated on the video 
tape. For all runs, a count of the total number of blinks was made. Data 
for each subject and level of difficulty were obtained. These are shown in 
Table 14 and Figure 11 respectively. As can be seen, there is a tendency 
for blink rate to decrease as the level of task difficiilty increases. The ab- 
solute value of this difference is quite small, however. 

PUPIL DIAMETER 

Results of the general Stepwise Linear Regression Analysis presented ear- 
lier in this section (Table 8) indicated that pupil diameter was the one and 
only variable of all those studied which demonstrated a strong predictive 
relationship with pilot workload, that is, the Cooper-Harper ratings. There 
is evidence in the literature which supports this relationship. Bradshaw, 

1968, (Reference 2) found that pupil diameter increased both as a function 
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Subject 


1 


2 


3 


5 


1 

10.28 

10.92 

11.36 

9.70 

10.08 

2 

3.36 

3.68 

3.56 

2.24 

2.40 

3 

10.80 

9.60 

7.00 

8.00 

7.80 

4 

1.76 

2.36 

2.24 

1.52 

1.44 

5 

2.08 

2.68 

2.56 

2.28 

3.20 

6 

11.04 

12.28 

11.88 

10.56 

14.08 

7 

9.16 

18.08 

9.40 

9.24 

7.88 

8 

5.52 

4.72 

5.12 

4.16 

3.64 

Average 

6.66 

7.04 

6.64 

5.96 

6.32 
















Blink Rate (number oer run) 



Task Difficulty 


Figure 11. Average Number of Blinks Per Run 
as a Function of Task Difficulty. 
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J 


of rate of stimulus presentation and task complexity in a task involving the 
continuous processing of alphanumeric information, Simpson and Hale, 1969, 

i 

(Reference 8) found similar effects in a decision-making task. Payne, et. al,, 
1965, (Reference 6) and Hess and Polt, 1967, (Reference 3) found that pupil 
diameter increased as the difficulty of a mental arithmetic problem increased. 
These studies are examples of previous research concerned with the rela- 
tionship between pupil diameter and task difficulty. The positive results ob- 
tained suggested that this variable might be of value in the present analysis. 

The results of the general stepwise linear regression analysis reported 
above (See Table 8) supported this hypothesis. 

In Table 15, average pupil diameter is presented by subject as a function of 
flight segment (Event) and difficulty level. Average data across subjects 
are presented in Figure 12. In spite of a rather large variability between 
subjects, trends can be seen within subjects. Pupil diameter tends to in- 
crease across events within a level although there is very little change 
across levels. While the magnitude of these changes averages only about 
two to four percent, the changes are rather consistent in direction. In fact, 

72 percent of all comparisons between Event 1 versus Event 3 and Event 3 
versus Event 5 showed an increase. 

Earlier research (Reference 3, Hess and Polt, 1964) bad reported changes 
in pupil diameter as large as 30 percent over a baseline condition. The task 
used in this study (mental arithmetic) lasted only a few seconds, as compared 
with the present data which were averaged over as much as five minutes. It 
was possible, therefore, that the small changes observed were due to averaging 
over prolonged time intervals. It was also possible that pupil diameter 
changes would be most marked in reaction to increases or decreases in 
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Pupil Diameter (millimeters) 



Task Difficulty 


Figure 12. Average Change in Pupil Diameter as a Function of 
Task Difficulty 
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error on one of more of the state variables. To investigate this, individual 
run data were examined by preparing short time history plots for each of 
the 200 runs. In each plot, state variable error and pupil diameter were 
plotted as a function of time into run. An example of two such plots, one 
for Level 1 and one for Level 5, are presented in Figure 13 which shows the 
relationship between pupil diameter changes and changes in error for command 
airspeed, localizer, and glideslope. Each of these plots was examined by 
eye to determine if a more comprehensive analysis was warranted. 

While marked fluctuations in pupil diameter were observed over time, there 
appeared to be no consistent relationship between state variable error and 
pupil diameter. This conclusion was supported by a similar lack of corre- 
lation between instrument dwell time and pupil diameter at the more 
general level of analysis reported earlier in Table 7, which was based 
on data averaged over entire runs. 

Calculations of pupil diameter are slightly affected by eye rotation relative 
to the mirror system in the oculometer. To determine if this change was 
larger than expected and a source of confounding, pupil diameter was cal- 
culated individually for each instrument. This was done by calculating 
average pupil diameter over all fixations on a given instrument. Differences 
were observed among instruments but again the differences were small (less 
than one millimeter) and not consistent in magnitude or direction. 

Even though pupil diameter averages did not vary widely, additional analyses 
were performed using ADI-related pupil diameter data. The ADI was chosen 
because a great majority of all fixations were on that instrument. In Figure 
14, pupil diameter is plotted for two different subjects, one run at Level 1 
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A. Data for Level 1 (Low Workload) 
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B, Data for Level 5 (High Workload) 


Figure 13. Short-time History Plot of Errors for Three State Variables 
and Pupil Diameter, Data for Subject 2. 







PUPIL DIAriETER (mm) 
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Figure 14, Pupil Diameter Data for Individual Runs 
by Individual Subjects at Two Levels of 
Task Difficulty. Data are Calculated 
only during Fixations on the ADI, 
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and one at Level 5 for each. The data were calculated from samples of ADI 
fixations only and are averaged over 30^-second intervals. The plots show a 
trend toward larger pupil diameter for the more difficult Level 5 runs and 
a slight trend toward larger pupil diameter as time into run increases. How- 
ever, as was found with the average data reported earlier, the changes are 
extremely small. 

As a final attempt at understanding the nature of the pupil diameter variable, 
a series of Stepwise Linear Regression Analyses were computed. The de- 
pendent variables were pupil diameter measures calculated individually for 
each of the ten instruments. Data were averaged over 30 -second time in- 
tervals for each run. As was done with the dwell time data, regression 

analyses were performed for individual subjects and for all subjects over 

2 

various time segments. The resulting R values were extremely low for 
all tests, typically around 0.25 and none as high as 0,40. Thus, when seg- 
mented by instrument and averaged over short time intervals, pupil dia- 
meter accounted for less than 40 percent of the variance in the Cooper- 
Harper ratings. 

PUPIL DIAMETER RE-EXAMINED 

The closer examination of pupil diameter was initially undertaken to clarify 
the nature of its relationship to pilot workload. In light of that objective, 
the above results were quite unexpected. How can the conflicting findings be 
reconciled? Why was the strong relationship with workload lost when the 
pupil diameter data were partitioned by instrument? 


62 



The most reasonable explanation that can be offered at this point is that two 
different variables were being examined in the two different sets of regres- 
sion analyses. In the partitioned set performed last, it is probable that a 
"true" measure of pupil diameter was being evaluated. It showed no strong 
relationship to workload. 

The initial regression analysis which showed the strong relationship between 
pupil diameter and workload was most likely not a pure measure of pupil 
diameter. It is suggested that this general variable pooled across instru- 
ments and averaged over entire runs is a composite variable reflecting both 

! 

actual pupil aperture and some measures related to relative time spent on 
the various instruments. For example, the image of the pupil as sensed by 
the oculometer becomes slightly eliptical as the eye rotates away from center. 
Thus dwells on peripheral instruments would produce distorted measures of 
pupil diameter. This fact combined with actual changes in pupil size might 
result in some systematic "workload" related trend. 

If this argument is valid, and the variable is not in fact purely pupil diameter, 
it in no way diminishes its potential value. On the contrary, it should be 
noted that this more general measure indicates a relationship that neither 
dwell time data or instrument -specific pupil diameter alone demonstrated. 

PILOT PARAMETER ESTIMATION 

In this sub-section, results of an independent analytical effort performed 
under this contract are discussed. The oculometer data were used as the 
basis for selecting samples of data from within a run. Interest was focused 
on the ADI since it provides so much flight -related data on one instrument. 
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In part, the analysis concentrates on determing what information the pilot 
is usingt^as he dwells for prolonged periods on the composite flight director. 


Maximum Likelihood Estimation (MLE) software was used to estimate pilot 
parameters with established model structures. These estimates were re- 
ferenced to the eye location, as supplied by the oculometer, to determine 
the characteristic pilot behavior for various tasks associated with landing 
approach. Some theoretical considerations of MLE are presented in Ap- 
pendix C . 

A brief history of pilot modeling follows. 

"Crossover" Pilot Model 


Pilot modeling using parameter fits to pre -defined dynamic structures has 
been developed over a number of years. McRuer and Krendel, 1957 (Ref- 
erence 4) developed the initial model based upon cascaded "classical" 
dynamic elements defined in the frequency domain. This model can be 
described as follows for single -axis compensatory tasks: 


ref 


where 



Np (to) = the pilot describing function 

y _ = the commanded track state 
•^ref 

y = the actual track state 
e = the displayed track error 


(3) 
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The describing function, N^(tu), is 

, , , , <^Ld®«> -TS 

° ’^p (t; s+1) ® 

Lg 


( 4 ) 


T = computational delay ( fixed and known) 


T_ = neuromuscular lag 

Lg 

= pilot lead compensation 


Kp = pilot gain 


The parameters provide the key clues to pilot workload and stress. The 
computational delay, t, is variable in human beings but is fairly consistent 
for trained pilots. t= 0.2 second is sufficient for most purposes. 


The neuromuscular lag,. T , varies with task difficulty: a large value in- 

Lg 

dicat ea an easy task, that is, pilots can be a little lazy, and a small value 
represents more difficult tasks. 

The lead time constant, T , represents the pilot's attempt to differentiate 

LfCl 

a displayed quantity. This occurs for plants that exhibit larger than first 
order roll off characteristics at the plant/pilot crossover region (i.e. , more dif- 
ficult tasks). 


The pilot will adjust his gain, K^, to obtain a -i frequency response slope 
at the crossover point, that is, he likes to have the pilot plant system be- 
have as an integrator at the dominant control task frequency. 
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Optimal Pilot Model 

The use *of optimal control theory (Reference 1, Baron, et. al., 1970) is a 
method of estimating pilot dynamic behavior using time measurements as 
opposed to the frequency domain approach. Based upon the argument that 
the pilot attempts to optimize his performance for a given task, the use of 
the optimal control approach extends the classical model in a number of 
areas: 

• explicit structure to multi input /multi -out put systems 

• more explicit treatment of scanning behavior, task interference, 
and operator workload 

• state -space formulation to allow application of powerful numer- 
ical algorithms to obtain desired parameters efficiently 

• theoretical noise treatment embedded into the formulation 
Optimal "Estimated" Pilot Model 

The use of parameter estimation techniques such as msucimum likelihood 
(Reference 7, Phatak , 1975) allows further refinement to the optimal model 
by reducing the number of arbitrary parameters to be chosen, quadratic 
performance index weights, for example. 

There are some conflicting arguments between these last two techniques 
(Reference 7, Phatak, 1975) but they are mostly based on heuristic assump- 
tions made by each model. Results for both are good. 
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Parameter Estimation with Scanning Information 


The data created using the oculometer provide a unique set of information 
for looking at pilot dynamic behavior. Previous studies (For example, Reference 7 
Phatak, 1975) did not have the luxury of knowing exactly where the pilot 
was looking on the instrument panel. 

The earlier models discussed use an additive noise term to account for 
scanning. Sometimes called a pilot remnant, the effect is to degrade 
primary axis tasks with assumed scanning behavior. 

For the present study, the certain knowledge of pilot attention allows removal 
of remnant over a given period of time. 

Pilot Model Structure 

As described in Appendix C,a state space form of the model is used in the 
identification. Figure 15 shows the model employed in the current analysis. 

This structure was used because it has the potential for reconstructing 
Equation (4) for each input/ output pair. In continuous state space, this 
model would be 

X = Fx 

y = Hx + Du + I (6) 

where; 

F L 


Lg 


(scalar) 
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1 



= T 


Lg 


K 




K 

gSi 


°2 i M 

H A [l] 


(identity matrix) 
(scalar) 


D A 

T 

X 


■ScsP,- 

= (e, V, H, HSP, GS) 



y A 6 e 

Tj A observation noise ~ N*(0, white 

5 A motor noise ~N (0, ct ) white 
- m 


The MLE software descretizes this model for calculations and converts fi- 
nal results back into continuous form. 


RESULTS OP MAXIMUM LIKELIHOOD ANALYSIS 


Results were generated for one pilot flying through the sequence of five 
events from probelm entry to touchdown. The sequences were chosen from 
oculometer results which indicated sole concentration on the ADI, 


The initial observation is that in all cases the time constant parameter ap- 
proached large values. This forced the lag term into an integrator. The 
gain values, therefore, reflected relative amounts of proportional plus 
integral control, used by the pilot for a given input variable. 

N(m, ct) = normal distribution with mean 'm' and variance 'ct 
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I 

The most revealing results consisted of examing the weighted residual por- 
tion of the likelihood function (Appendix C). 

Let: 

R = E? . v.pv; (7) 

1=1 1 1 

where ^ = apriori residual covariance matrix 

V. = 6e - 6e 
1 a p 

6e„ = Actual Fore -aft Stick Position 
a 

'' 6e = Predicted Stick Position of the Identification (at the ith time 

P 

epoch) 

This response gives a good indication of the ability of the identification to 
reconstruct the actual stick position. 

To find the dominant variables in the system, gains were sequentially set to 
zero and predicted stick position, 6e^, was recalculated without the benefit 
of the entry of a given variable into the system. Critical inputs would natu- 
rally have a larger impact upon the residual response, R; that is, the larger 
the deteriorization of the response implies, the more important gain value 
is to the reconstruction. 

Table 16 displays the results of this sensitivity analysis. Specific conclu- 
sions are: 

1, Results correlate well with the overall control task, i,e,, the 
the dominant variables were the important ones for specific con- 
trol tasks. 


70 



Table 16. Likelihood Function Parameter Sensitivity 



O' 

—1 

Lu a. 

ADI dwell 1 

ADI dwell 2 

ADI dwell 3 

ADI dwell 4 

AOI dwell 5 

ADI dwell 6 

ADI dwell 7 

AOI dv.-oll 8 

None 

2.0% 

3.46 

57.72 

42.79 

7.73 

1.78 

16.9 

.0785 


424.3 

107.3 

3.15X10^ 

7.00X10® 

7.02X10® 

3.53X10® 

805.3 

6.70X10® 

“^GSl 

Kvi 

'^Hl 

^El* 
)( * 

'^GS2 

not used 
1008.9* 

not used 
60.86 

1.4X10^* 

7.14X10^ 

7.69X10®* 

7.40X10^ 

7.82X10^ 

6.61X10® 

13.0 

388.8 

1.82X10^* 

1 . 35 x 10 '* 

1.56X10®* 

4.28X10® 

157.0 

10.77 

1.39X10^* 

2.16X10®* 

2.03X10^ 

890.1 

1.57X10^* ■ 

1.24X10®* 

2.15 

3.46 

1.27X20^ 

6.56X10®* 

7.88X10® 

2.81X10® 

• 1.97X10^ 

16.9 

119.9 

54.50 

1.78X10^ 

4.90X10® 

3.06X10® 

1.06X1 o'** 

5.31X10® 
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* Indicates variable display on ADI 

★ 

* Recognized as having a relative major influence 

Variable definitions: = Lag time Constant; GS = Row glide slope; v - Velocity; H = Altitude; E = Horizontal Steering pointer 





























2. Higher impact from the horizontal steering pointer might have 
been expected ] however, in general it does not greatly affect 
the identification relative to other variables, 

3. Some shift of emphasis can be noticed from integral to propor- 
tional influences as the approach proceeds in time. This pos- 
sibly indicates a shift in workload, that is, integral control im- 
plies a less difficult task. 

Also shown in Figures 16 through 22 are comparison plots of actual stick 
position and generated stick position based upon identified parameters. The 
actual plots, shown on top of each figure, contain a 12 cycle/second nuisance 
which, although low -passed before analysis, still shows up. The recon- 
structed stick position, however, should not and does not reflect this as no 
provision in the model was made to account for this. 


Some open questions: 

1) Do the influential variables imply that the pilot actually picked 
up peripheral information (altitude and glideslope)? 

2) Is the reason for their high impact related to the overall control 
task and not the- display layout? 

The second alternative seems more likely. The fact that the ADI dwell is 
predominant implies that attention is given to the horizontal steering pointer. 
The pilot can get the appropriate path control information from this variable. 
Hence, he is controlling altitude and glideslope from the pointer. 
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Figures 16 through 22, referenced on page 72, 
follow on the next 14 pages. The plots (actual 
versus generated) for each ADI Dwell appear 
on facing pages. 
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SECTION VI 

I f 

CONCLUSIONS AND RECOMMENDATIONS 


The overall objective of the study was to develop a preliminary model of 
workload comprised of eye behavior variables. To do this, rate of change 
of the state variables was assumed to be the best indication of the ordering 
of the flight conditions in terms of task difficulty. Eye data were then ex- 
amined to determine how changes in the various vision -related variables 
reflected this difficulty. 

The primary tool used in the development of the model was stepwise linear 
regression. At a general level of analysis, pupil diameter was shown to 
have a strong relationship to pilot workload. This was not found when the 
data were examined in more detail using individual runs for individual sub- 
jects. It was concluded that the pupil diameter measure as used initially 
was most likely not a pure measure but rather a composite score reflecting 
both pupil dilation and instrument scanning activities. If this is in fact true, 
it does not diminish the value of the obtained relationship. 

Extensive analysis of dwell time on specific instruments showed high varia- 
bility between pilots and within a given pilot over several runs. No signifi- 
cant relationship between instrument dwell time and pilot workload was 
found. This was somewhat unexpected. 

Recommendations for future research include the examination of the pupil 
diameter variable from other data sets such as those being obtained at the 
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Langley Research Center, Simulation and Human Factors Branch. Dwell 
time data should be similarly explored. H is recommended that emphasis 
by placed on examining more extensive data from perhaps fewer pilots. No 
consistent "strategy" was apparent over several runs under equivalent con- 
ditions. If such a strategy exists, it may become more apparent as more 
data are examined. 

While the present study included what was felt to be a reasonable range of 
task difficulty, it might be of value to explore a greater range of conditions, 
particularly at higher levels of pilot workload. It may be found that certain 
relationships appear only under the more extreme conditions . 

The use of maximum likelihood techniques to identify dominant pilot system 
variable inputs, such as variables of primary in^jortance to the given pilot 
task, proved successful. Furthermore, positive indications of the ability 
of the horizontal steering pointer to provide this information were obtained. 
This is because fixations on the ADI could be isolated by using the oculome- 
ter output. 

On the other hand, the MLE results proved somewhat disappointing with re- 
gard to the original analysis goals. The MLE process did not isolate the 
display variable of concentration. For example, HSP fixations would be 
expected to produce higher dominance of this display variable in the identi- 
fication. This was not the case as other variables, usually more closely 
related the actual control task, tended to dominate the pilot parameter re- 
construction. 
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Recommendations for expansion of this technique are as follows: 

1. A more precise definition of the d 3 mamic characteristics of the 
pilot could be obtained by incorporating several modifications to 
the analysis process. 

a) Only a single display variable should be used in the identification 
process. In the present case, only the horizontal steering poin- 
ter should be used. This would eliminate gain identification on 
other display variables resulting in computation reduction and 
would lead to expansion of pilot model stmcture. 

b) Since the horizontal steering pointer is a display design variable, 
that is, frequency- shaped combination of plant -states, the de- 
sign process can be optimized by examining pilot parameters as- 
sociated with various designs. These parameters can be eval- 
uated for pilot acceptability. 

c) A better way of normalizing display variables should be pro- 
vided. This will produce easier interpretation of pilot parame- 
ters. The scaling used here produced some large parameter 
magnitude differences. 

2. Pilot parameters correlated well with pilot ratings under ideal cir- 
cumstances. Under real world flying conditions, the oculometer is 
an effective tool for isolating specific periods for close examina- 
tion. 

3 . Finally, it should be recognized that this MLE effort was very 
limited in scope. Analysis of more samples representing a variety 
of flight segments and task difficulty should be considered. 
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APPENDIX A 


SIMULATION OF THE BOEING 737 A /C IN THE 
SIGMA FIVE HYBRID SYSTEM 


SUMMARY 

This section summarizes the 737 A/C simulation operational in the XDS 
Sigma Five Computer. The simulation was configured for approach and 
landing (i.e., Mach < 0.35, altitude <1524 meters) and included the nonlinear 
A/C coefficients associated with large attack and flap angles. The simula- 
tion made use of the General Purpose A/C Simulation System (GPASS) 
using the THRUST format developed by Honeywell. 

The six degree-of-freedom software package (GPASS) was used in the 
development of the hybrid simulation for this study. The mass aero- 
dynamic and inertial parameters of the Boeing 737 twin jet airliner were 
used. Aircraft coefficients were evaluated for high attack angle conditions 
where Mach and altitude were less than 0.35 and 1524 meters, respectively 
since the flight segments to be studied were terminal approach and landing. 

An Instrument Landing System (ILS) simulation was used which computed 
angular error signals from ILS beam centers along its reference axes 
to aircraft position. This angular error set was transformed to a rectangular 
set so that errors had the units of feet rather than degrees. The computation 
was: 

« 

= R sin (GS) 
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where 


e is elevation error above or below glide slope, in meters, 

R is range from transmitter, in meters 
GS is angular glide slope error, in degrees 

= R sin (LOG) 

where 

e . „ is azimuth error to the right of runway centerline, in meters 
R is range from transmitter, in meters 
LOG is angular localizer error, in degrees 


This avoids the problem of great increases in the sensitivity of the angular 
signals during the final approach phase when range to touchdown approaches 
small values. This is an alternate to course softening which uses a variable 
gain on the error signals as altitude values decrease near touchdown. 


The glide slope and localizer error signals were displayed to the pilot as 
raw data. The elevation and azimuth error signals were used in the guidance 
calculations of the command /steering signals to the flight director. The 
stabilization portion of the command/steering signals used compensated 
vehicle attitude terms. 


The command laws used for the steering bars were 




A-2 



where 


HSP is the Horizontal Steering Pointer signal to the ADI 
(scale factor at 0.0254 cm/Volt), 

is the glide slope error, in meters, 

S is the LaPlace operator, 

9 is the aircraft pitch attitude, in degrees 


9^ is pitch bias 


vsp= -1.0 [0.30 stHrr 


where 


VSP is the Vertical Steering Pointer signal to the ADI 
(scale factor at 0.0254 cm/Volt) 


®AZ localizer error, in meters 

0 is aircraft roll attitude, in degrees 


A speed command was developed and displayed to the pilots on a speed 
bug on the left side of the ADI, The commands were; 

V = 200 knots constant, R > 12000 
c 

V^ = 0. 0038 R knots, 12000 > R > 6000 
V^ =120 knots, R < 6000 to touchdown 

During the approach it was possible to fly out of the glide slope beam. 

The system logic placed the horizontal steering pointer in the stow position 
when this occurred. 



A flare command was developed to drive the HSP rather than glide slope 

I 

error at the terminal end of the final approach. At altitude equal to 

flare was initiated through a 2. 5 second fader at the same time ILS 
was faded out. The flare command was 

HSP = H + 6h 
where 

h is aircraft altitude, in meters 
h is descent rate, meters per second 

SIMULATION DESCRIPTION 

Basic mass aerodynamic and inertial parameters of the Boeing 737 are 
shown in Table A-1. Figure A-1 is a three-sided view of the aircraft. Longi- 
tudinal control is through the elevators and horizontal stabilizer. The 
stabilizer is primarily used for trim, while the elevators, mounted on the 

I 

aft end of the stabilizer, are used for maneuvering and damping. Lateral 
control is achieved through use of ailerons, spoilers, and rudder. The 
spoilers are used both for assisting the ailerons and providing speed 
braking. 

Additional surfaces are available for high-lift conditions. These surfaces 
are trailing and leading edge flaps as well as leading edge slots. Character- 
istics of all the above mentioned surfaces are discussed in more detail in 
Section Vn. The project for which this simulation was developed had 
as its purpose to evaluate pilot workload during approach and landing. 

Hence A/C coefficients were evaluated for high attack-angle conditions. 
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TABLE A-1. SUMMARY OF AREAS AND DIMENSIONS 


Symbol 

Item 

Value 

Dimension 

S 

Wing area 

91.04 

2 

m 

c 

Wing mean aerodynamic chord 

3.41 

m 

b 

Wing span 

28,35 

m 


Effective engine moment arms 
about c. g. 



Ye 

Lateral arm 

4.94 

m 

^E 

Vertical arm 

1.52 

m 


Wheel base 

10.46 

m 


Wheel tread 




Main gear to c point 

0.98 

m 


Mass (weight) 

2797 slugs 



hcK 

3.75 . 10^ slug 



hY 

8.75 • 10^ slug 



^ZZ 

1. 20 • 10^ slug 



Ixz 

ScY " Vz ^ ° 

4. 80 • 10^ slugs 
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Figure A-2 is a functional block diagram illustrating the operation of the 
control system. Manual/SAS interaction is fairly standard with the exception 
that a portion of the elevator position is a function of stabilizer trim position. 

Figure A-3 is a listing of the aerodynamic routine, illustrating the aero 
dimensionless coefficients used in the simulations, as well as their method 
of calculation. Since the flaps have a strong effect during landing, the 
aero coefficients are sensitive to flap position. 

The simulation is connected to the cockpit simulator via the trunk inter- 
connect rack. Figure A-4 is a schematic of the cockpit controls and hybrid 
link. Included are pitch, roll, rudder pedals, throttles, spoiler, and flap 
control. Figure A-5 is a schematic of the cockpit instruments, driven 
from the hybrid link. These instruments include a flight director and ILS 
command needles, compass, altimeter, attitude rate, engine thrust and 
flap angle, and indicated airspeed. 

Scale factors for the flight director and compass are 1, 8° per volt (180° = 

100 V). Altitude rate, attitude, and indicated airspeed are given by the 
following equations: 

h(v) = 2. 7 + 0.0396 • h(m/sec)* 

lAS(V) = 3.25 - 0.0165 • V(knots) + 9 • V^(knots) 

CAS CAS 

h(course/fine) 

h = 2.3 + 0.0004 . h(m) 
c 

h. = -96 + 0, 045 ■ h(m) 

I « 
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0.1 deg/deg 


*( 


0.15 deg/deg 



Figure A -2. Functional Block 
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/A(37a) ), (CMOS /A(3751) 

/ ICMDF 
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Figure A -3. Listing of the Aerodynamic Routine 
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lY 
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1.0 
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AL 
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V 5 
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76 



GO? 

£ 
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LT 

a 
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79 
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80 
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84 
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S3 



CZ2 

B 
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8 6 



TH 

0 
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87' 
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8 

SIN( TM) 

8S‘ 
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8 
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o9‘ 



SAL 

s 

SIN( AL) 

90 
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a 
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91' 



r> 

0 
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92' 



u 

c 

VEL*CAL 

93' 
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0 

94 
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a 

l.O/VcIL 

95 



SIG 

0 
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97' 
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98' 
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a 
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99' 
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a 
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100 
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e 
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c 
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Figure A -3. Listing of the Aerodynamic Routine (Continued) 
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1 bS 
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Figure A-3. Listing of the Aerodynamic Eoutine (Concluded) 
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Figure A -5. A Schematic of the Cc 
from the Hybrid Link 
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It should be noted that the attitude instrument requires a course and fine 
signal; in order to get realistic vertical speed, h, is filtered with a one- 
second lag. 

The longitudinal response to or gusts and a pilot input indicate that both free 
aircraft and SAS response is well damped. Predicted free aircraft response 
is: 

o)^= 1.8 rad /sec (3 -second period) 

Q = 0.45 

Observed period and damping ratio was about 2. 8 seconds at a damping ratio 
of 0.4. 

Response to pilot inputs for the lateral axis was improved by the SAS so 
that the response was well damped and maintained the command roll rate. 
Predicted lateral response is: 

ou = 1. 47 rad/sec 
P 

Cp = 0.14 

The observed frequency and damping ratio was 1. 54 rad/sec and 0. 13, 
respectively. 
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APPENDIX C 


MAXIMUM LIKELIHOOD ESTIMATION ANALYSIS PROCEDURE 


The procedure used in the analysis reported in Section V consisted of em- 
ploying Honeywell's Maximum Likelihood Estimation software package to 
simulation data generated during the initial phase of the Honeywell study. 
Use of the Oculometer in Pilot Workload Measurement. This package was 
developed on earlier Honeywell internal research efforts and refined under 
contract to NASA/Langley (Reference 10), 

Basically, the identification operates on an input /output data set with some 
notion that the two are related dynamically. A dynamic structure is first 

postulated: 

« 

X = f(x, t, u, c, I) ; x(o) = x^ 
y = h(x, t, u, c) + Tl 


where 

x(t) = n-dimensional state vector 
c = unknown constant parameters 


|(t) = white noise process 




E 


git) 


=0; E 


g^it)g'^(T} =Q6(t-T) 


u(t) = input vector 


y(t) = output vector 


T)(t) = measurement disturbance; white noise 

t' 


E 


TKt) 


= 0; E 


T](t) T1 (t) 


= R6(t-T) 
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Operating in a linear descrete format the model becomes 


X = A(c)x. + B (c)u. + B (c)§i 
1+1 1 1 1 2 

y. = C(c)x. + u. + Tli 
1 11 

The goal is to identify the unknown parameter vector c which best fits the 
model to a measured input time sequence, u.*s, and measured output time 
sequence, y.'s. 

MLE attacks the problem with a conditional density function 

P(a| b = p, or) = probability density function for random variable 
'a' given that random variable 'b' has a value 'P'; 
at is also given. 


In this case 


c = Arg 


= I, 


N = the entire collection of Y and U 


(Arg[max P] means the argument 
or variable of the P function which 
maximizes P. ) 


^2' ••• ■ J'n’ 


^2 


The interpretation is that § = c maximizes the probability that outputs 
will result from a system with as the input sequence. 


In usable form, the actual maximization is performed on the log likelihood 
function; 

L(|) = Ln p(Y Ic = I, U ) 

9 N 

= 1/2 ([ c - ? Ir P"^ - 1/2 E (Ln detp, , + f( V. |r.-l) 
o o , k ’ " k " p 
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where and V, are defined as follows : 
k k 



where y is the actual output measured at time (K) 

K. 

y is the predicted output of the model with parameters 

3__ is the error covariance of the residuals, E v 
K L ^ _ 

Further mention of the Kalman filter portion of this development is deferred 

(Reference 9, Tse, 1973) because as we will soon see, it is not used 

here. 


The use of the Kalman estimator assumes some knowledge of the noise char- 
acteristics Q and R (however these two can be identified). For the current 
application it was felt that these noise characteristics were not sufficiently 
known to either assume some Q and R or estimate them. This means that 
the identification proceeds without Kalman filtering, that is, no smoothing 
of measurements. The identification, therefore, can be reduced to a least 
squares on the residuals v. . 

PN ^ o 1 

c=Argminj^E^ llv^(§)|| 
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